In the most general case, speculative memory accesses are accesses made to the memory system by the processor that do not directly contribute to program output but are made in an attempt to speed up processor performance.
Speculative memory accesses are made for various purposes and are of many varieties including, but not limited to: prefetch memory accesses; wrong-path memory accesses; various memory accesses made while the processor is in a speculative state such as speculatively executing ahead for pre-fetching, also known as scouting, and/or any other memory accesses performed in an attempt to prefetch data from the memory system to the on-chip memory cache before the data is actually needed by the processor.
A problem with speculative memory accesses arises when a speculative access is made to a cache line that will not be used since, when this occurs, the speculative access then installs a cache line into the cache which forces another cache line to be removed from the cache. In a set-associative cache, this, in turn, changes the replacement ordering of other cache lines in the set and has significant detrimental effect on the system performance.
In particular, when a processor accesses a cache, if the cache line needed is not already present in the cache, the cache line is fetched from a lower level memory and then inserted into the cache. This process typically takes multiple clock cycles. The data is inserted at a specific cache line in the cache and any data already in that cache line is displaced out of the cache, back into a lower level memory directly, or by a reordering of another cache line within the cache.
If, as is often the case in modern system architectures, the cache is set-associative, then this displacement of the data in the cache line used for the speculative access causes a modification to the order in which the other cache lines will be displaced, once again causing a reordering and cache line displacement. If the speculative data is subsequently accessed, then the speculative memory access was a useful prefetch and the reordering is justified. However, if the data in one of the displaced cache lines is subsequently accessed, but is no longer in the cache due to displacement by the speculative access, then the speculative access is said to have caused a pollution. Significantly, a single speculative access may cause multiple pollution events in set-associative caches. Consequently, the disruption and detrimental effects of a single speculative access can be quite significant
As the speed/frequency of processors increases, the disparity between processor speeds and memory system speeds continues to grow. As a result, speculative memory access has come to play an increasingly critical role in ensuring program performance and allowing systems to take advantage of processor potential. Consequently, the problems created by speculative accesses to cache lines that will not be used is becoming more and more significant and can no longer be overlooked or accepted.
What is needed is a method and apparatus for tracking speculative accesses that cause prefetches and/or pollutions.